由于其低成本和快速移动性,无人驾驶汽车(UAV)现在已广泛应用于数据获取。随着航空视频量的增加,对这些视频自动解析的需求正在激增。为了实现这一目标,当前的研究主要集中于在空间和时间维度沿着卷积的整体特征提取整体特征。但是,这些方法受到小时接收场的限制,无法充分捕获长期的时间依赖性,这对于描述复杂动力学很重要。在本文中,我们提出了一个新颖的深神经网络,称为futh-net,不仅为整体特征建模,而且还模拟了空中视频分类的时间关系。此外,在新型融合模块中,多尺度的时间关系可以完善整体特征,以产生更具歧视性的视频表示。更特别地,FUTH-NET采用了两条道路架构:(1)学习框架外观和短期时间变化的一般特征的整体代表途径,以及(2)捕获跨任意跨越任意时间关系的时间关系途径框架,提供长期的时间依赖性。之后,提出了一个新型的融合模块,以时空整合从这两种途径中学到的两个特征。我们的模型对两个航空视频分类数据集进行了评估,即ERA和无人机操作,并实现了最新结果。这表明了其在不同识别任务(事件分类和人类行动识别)之间的有效性和良好的概括能力。为了促进进一步的研究,我们在https://gitlab.lrz.de/ai4eo/reasoning/futh-net上发布该代码。
translated by 谷歌翻译
开放式视频异常检测(OpenVAD)旨在从视频数据中识别出异常事件,在测试中都存在已知的异常和新颖的事件。无监督的模型仅从普通视频中学到的模型适用于任何测试异常,但遭受高误报率的损失。相比之下,弱监督的方法可有效检测已知的异常情况,但在开放世界中可能会失败。我们通过将证据深度学习(EDL)和将流量(NFS)归一化为多个实例学习(MIL)框架来开发出一种新颖的OpenVAD问题的弱监督方法。具体而言,我们建议使用图形神经网络和三重态损失来学习训练EDL分类器的区分特征,在该特征中,EDL能够通过量化不确定性来识别未知异常。此外,我们制定了一种不确定性感知的选择策略,以获取清洁异常实例和NFS模块以生成伪异常。我们的方法通过继承无监督的NF和弱监督的MIL框架的优势来优于现有方法。多个现实世界视频数据集的实验结果显示了我们方法的有效性。
translated by 谷歌翻译
我们提出了一种新颖的框架,以便长期分类大型时间序列数据。长时间序号(L-TSC)是一个具有挑战性的问题,因为DATAOFTEN包含大量无关的信息到Checlassification目标。无关期限会降低分类的性能,而系统相关性是未知的。本文提出了一个不确定性,意识到多个型号(MIL)框架,以识别最相关的初期性。预测性的不确定性使得设计指示迫使MIL模型从判别判别期间学习的Anittent机制。此外,预测的不确定性yields是一个原则性的估计器,以识别是否是预测的istrustworthy。我们还通过培训在其可用性上​​进行单独培训并进行不确定性意识融合表的最终预测来培训另一个模型ToAcModate不可靠的预测。系统评估是进行自动识别系统(AIS)数据,该数据是识别和跟踪现实世界船只的COL。经验结果DemonstriteStriteStepriteStepritisht,该方法可以基于轨迹有效地检测血管的阀门,以及与其他可用数据模块(在我们的实验中使用的合成 - ApertureerAdar或SAR图像)的不确定性令人遗憾地进一步改善检测精度。
translated by 谷歌翻译
用于图像分类的最可公开的数据集是单个标签,而图像在我们的日常生活中是固有的多标记。这种注释差距使得许多预先接受的单标准分类模型在实际情况下失败。该注释问题更加关注空中图像:从传感器收集的空中数据自然地覆盖具有多个标签的相对大的陆地面积,而被广泛可用的注释空中数据集(例如,UCM,AID)是单标记的。作为手动注释的多标签空中图像将是时间/劳动,我们提出了一种新的自我校正综合域适应(SCIDA)方法,用于自动多标签学习。 SCIDA是弱监督,即,自动学习多标签图像分类模型,从使用大量的公共可用的单一标签图像。为实现这一目标,我们提出了一种新颖的标签 - 明智的自我校正(LWC)模块,以更好地探索潜在的标签相关性。该模块还使无监督的域适配(UDA)从单个到多标签数据中可能。对于模型培训,所提出的型号仅使用单一标签信息,但不需要先验知识的多标记数据;它预测了多标签空中图像的标签。在我们的实验中,用单标签的MAI-AID-S和MAI-UCM-S数据集接受培训,所提出的模型直接在收集的多场景空中图像(MAI)数据集上进行测试。
translated by 谷歌翻译
人工智能通过许多令人印象深刻的应用深刻地彻底改变了药物化学领域,但是这些应用的成功需要大量具有高质量注释的培训样本,这严重限制了数据驱动方法的广泛使用。在本文中,我们专注于反应产量预测问题,该问题有助于化学家仅通过一些实验试验选择新的化学空间中的高收益反应。为了攻击这一挑战,我们首先提出了Metarf,这是一种基于注意力的随机森林模型,该模型专门针对少量产量预测,其中随机森林的注意力重量通过元学习框架自动优化,可以快速地进行优化适合预测新试剂的性能,同时还提供了一些其他样品。为了提高少量学习绩效,我们进一步引入了基于尺寸的采样方法,以确定要进行实验测试然后学习的有价值的样品。我们的方法在三个不同的数据集上进行了评估,并在几乎没有预测上获得了令人满意的性能。在高通量实验(HTE)数据集中,我们方法论的前10个高收益反应的平均产量相对接近理想的产量选择结果。
translated by 谷歌翻译
人工智能,特别是通过深度学习的最新进步,在自然语言处理和计算机视觉等领域的许多任务中都取得了出色的表现。除了理想的评估指标外,这些模型通常需要高水平的解释性。因此,对模型将其输入映射到其输出的过程的解释是备受追捧的。不幸的是,机器学习模型的当前黑匣子性质仍然是一个尚未解决的问题,这种性质使研究人员无法学习并为模型的行为和最终预测提供阐释描述。在这项工作中,我们提出了一个利用对抗性逆强化学习的新颖框架,该框架可以为通过强化学习模型做出的决策提供全球解释,并捕获模型通过总结模型的决策过程所遵循的直觉趋势。
translated by 谷歌翻译
Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samples are not reliable for ignoring important clustering information. To solve these problems, we propose a Cluster-guided Contrastive deep Graph Clustering network (CCGC) by mining the intrinsic supervision information in the high-confidence clustering results. Specifically, instead of conducting complex node or edge perturbation, we construct two views of the graph by designing special Siamese encoders whose weights are not shared between the sibling sub-networks. Then, guided by the high-confidence clustering information, we carefully select and construct the positive samples from the same high-confidence cluster in two views. Moreover, to construct semantic meaningful negative sample pairs, we regard the centers of different high-confidence clusters as negative samples, thus improving the discriminative capability and reliability of the constructed sample pairs. Lastly, we design an objective function to pull close the samples from the same cluster while pushing away those from other clusters by maximizing and minimizing the cross-view cosine similarity between positive and negative samples. Extensive experimental results on six datasets demonstrate the effectiveness of CCGC compared with the existing state-of-the-art algorithms.
translated by 谷歌翻译
To generate high quality rendering images for real time applications, it is often to trace only a few samples-per-pixel (spp) at a lower resolution and then supersample to the high resolution. Based on the observation that the rendered pixels at a low resolution are typically highly aliased, we present a novel method for neural supersampling based on ray tracing 1/4-spp samples at the high resolution. Our key insight is that the ray-traced samples at the target resolution are accurate and reliable, which makes the supersampling an interpolation problem. We present a mask-reinforced neural network to reconstruct and interpolate high-quality image sequences. First, a novel temporal accumulation network is introduced to compute the correlation between current and previous features to significantly improve their temporal stability. Then a reconstruct network based on a multi-scale U-Net with skip connections is adopted for reconstruction and generation of the desired high-resolution image. Experimental results and comparisons have shown that our proposed method can generate higher quality results of supersampling, without increasing the total number of ray-tracing samples, over current state-of-the-art methods.
translated by 谷歌翻译
Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.
translated by 谷歌翻译
Representing and synthesizing novel views in real-world dynamic scenes from casual monocular videos is a long-standing problem. Existing solutions typically approach dynamic scenes by applying geometry techniques or utilizing temporal information between several adjacent frames without considering the underlying background distribution in the entire scene or the transmittance over the ray dimension, limiting their performance on static and occlusion areas. Our approach $\textbf{D}$istribution-$\textbf{D}$riven neural radiance fields offers high-quality view synthesis and a 3D solution to $\textbf{D}$etach the background from the entire $\textbf{D}$ynamic scene, which is called $\text{D}^4$NeRF. Specifically, it employs a neural representation to capture the scene distribution in the static background and a 6D-input NeRF to represent dynamic objects, respectively. Each ray sample is given an additional occlusion weight to indicate the transmittance lying in the static and dynamic components. We evaluate $\text{D}^4$NeRF on public dynamic scenes and our urban driving scenes acquired from an autonomous-driving dataset. Extensive experiments demonstrate that our approach outperforms previous methods in rendering texture details and motion areas while also producing a clean static background. Our code will be released at https://github.com/Luciferbobo/D4NeRF.
translated by 谷歌翻译